<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Feature Matching + Homography to find Objects</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../db/d27/tutorial_py_table_of_contents_feature2d.html">Feature Detection and Description</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Feature Matching + Homography to find Objects </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter,</p><ul>
<li>We will mix up the feature matching and findHomography from calib3d module to find known objects in a complex image.</li>
</ul>
<h2>Basics </h2>
<p>So what we did in last session? We used a queryImage, found some feature points in it, we took another trainImage, found the features in that image too and we found the best matches among them. In short, we found locations of some parts of an object in another cluttered image. This information is sufficient to find the object exactly on the trainImage.</p>
<p>For that, we can use a function from calib3d module, ie <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780" title="Finds a perspective transformation between two planes. ">cv.findHomography()</a></b>. If we pass the set of points from both the images, it will find the perspective transformation of that object. Then we can use <b><a class="el" href="../../d2/de8/group__core__array.html#gad327659ac03e5fd6894b90025e6900a7" title="Performs the perspective matrix transformation of vectors. ">cv.perspectiveTransform()</a></b> to find the object. It needs atleast four correct points to find the transformation.</p>
<p>We have seen that there can be some possible errors while matching which may affect the result. To solve this problem, algorithm uses RANSAC or LEAST_MEDIAN (which can be decided by the flags). So good matches which provide correct estimation are called inliers and remaining are called outliers. <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780" title="Finds a perspective transformation between two planes. ">cv.findHomography()</a></b> returns a mask which specifies the inlier and outlier points.</p>
<p>So let's do it !!!</p>
<h2>Code </h2>
<p>First, as usual, let's find SIFT features in images and apply the ratio test to find the best matches. </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">from</span> matplotlib <span class="keyword">import</span> pyplot <span class="keyword">as</span> plt</div><div class="line"></div><div class="line">MIN_MATCH_COUNT = 10</div><div class="line"></div><div class="line">img1 = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;box.png&#39;</span>,0)          <span class="comment"># queryImage</span></div><div class="line">img2 = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;box_in_scene.png&#39;</span>,0) <span class="comment"># trainImage</span></div><div class="line"></div><div class="line"><span class="comment"># Initiate SIFT detector</span></div><div class="line">sift = cv.SIFT_create()</div><div class="line"></div><div class="line"><span class="comment"># find the keypoints and descriptors with SIFT</span></div><div class="line">kp1, des1 = sift.detectAndCompute(img1,<span class="keywordtype">None</span>)</div><div class="line">kp2, des2 = sift.detectAndCompute(img2,<span class="keywordtype">None</span>)</div><div class="line"></div><div class="line">FLANN_INDEX_KDTREE = 1</div><div class="line">index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)</div><div class="line">search_params = dict(checks = 50)</div><div class="line"></div><div class="line">flann = <a class="code" href="../../dc/de2/classcv_1_1FlannBasedMatcher.html">cv.FlannBasedMatcher</a>(index_params, search_params)</div><div class="line"></div><div class="line">matches = flann.knnMatch(des1,des2,k=2)</div><div class="line"></div><div class="line"><span class="comment"># store all the good matches as per Lowe&#39;s ratio test.</span></div><div class="line">good = []</div><div class="line"><span class="keywordflow">for</span> m,n <span class="keywordflow">in</span> matches:</div><div class="line">    <span class="keywordflow">if</span> m.distance &lt; 0.7*n.distance:</div><div class="line">        good.append(m)</div></div><!-- fragment --><p> Now we set a condition that atleast 10 matches (defined by MIN_MATCH_COUNT) are to be there to find the object. Otherwise simply show a message saying not enough matches are present.</p>
<p>If enough matches are found, we extract the locations of matched keypoints in both the images. They are passed to find the perspective transformation. Once we get this 3x3 transformation matrix, we use it to transform the corners of queryImage to corresponding points in trainImage. Then we draw it. </p><div class="fragment"><div class="line"><span class="keywordflow">if</span> len(good)&gt;MIN_MATCH_COUNT:</div><div class="line">    src_pts = np.float32([ kp1[m.queryIdx].pt <span class="keywordflow">for</span> m <span class="keywordflow">in</span> good ]).reshape(-1,1,2)</div><div class="line">    dst_pts = np.float32([ kp2[m.trainIdx].pt <span class="keywordflow">for</span> m <span class="keywordflow">in</span> good ]).reshape(-1,1,2)</div><div class="line"></div><div class="line">    M, mask = <a class="code" href="../../d9/d0c/group__calib3d.html#ga4b3841447530523e5272ec05c5d1e411">cv.findHomography</a>(src_pts, dst_pts, cv.RANSAC,5.0)</div><div class="line">    matchesMask = mask.ravel().tolist()</div><div class="line"></div><div class="line">    h,w,d = img1.shape</div><div class="line">    pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)</div><div class="line">    dst = <a class="code" href="../../d2/de8/group__core__array.html#gad327659ac03e5fd6894b90025e6900a7">cv.perspectiveTransform</a>(pts,M)</div><div class="line"></div><div class="line">    img2 = <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaa3c25f9fb764b6bef791bf034f6e26f5">cv.polylines</a>(img2,[np.int32(dst)],<span class="keyword">True</span>,255,3, cv.LINE_AA)</div><div class="line"></div><div class="line"><span class="keywordflow">else</span>:</div><div class="line">    <a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>( <span class="stringliteral">&quot;Not enough matches are found - {}/{}&quot;</span>.format(len(good), MIN_MATCH_COUNT) )</div><div class="line">    matchesMask = <span class="keywordtype">None</span></div></div><!-- fragment --><p> Finally we draw our inliers (if successfully found the object) or matching keypoints (if failed). </p><div class="fragment"><div class="line">draw_params = dict(matchColor = (0,255,0), <span class="comment"># draw matches in green color</span></div><div class="line">                   singlePointColor = <span class="keywordtype">None</span>,</div><div class="line">                   matchesMask = matchesMask, <span class="comment"># draw only inliers</span></div><div class="line">                   flags = 2)</div><div class="line"></div><div class="line">img3 = <a class="code" href="../../d4/d5d/group__features2d__draw.html#ga62fbedb5206ab2faf411797e7055c90f">cv.drawMatches</a>(img1,kp1,img2,kp2,good,<span class="keywordtype">None</span>,**draw_params)</div><div class="line"></div><div class="line">plt.imshow(img3, <span class="stringliteral">&#39;gray&#39;</span>),plt.show()</div></div><!-- fragment --><p> See the result below. Object is marked in white color in cluttered image:</p>
<div class="image">
<img src="../../homography_findobj.jpg" alt="homography_findobj.jpg"/>
<div class="caption">
image</div></div>
 <h2>Additional Resources </h2>
<h2>Exercises </h2>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
